Many online systems allow for user interaction through natural language input (e.g., spoken word). For example, an online system may query a user for information, to which the user may respond using natural language, which may be transcribed into text that may be parsed by the online system to determine a response. However, automatic speech recognition and transcription software may not always be able to accurately transcribe spoken utterances received from users. Human analysts may be used to supplement automatic transcriptions of spoken utterances, but may not always be able to perform transcriptions quickly enough such that the online system can respond to the user in real time.